Skip to content

fix(dev): read the usage ledger where the host writes it (#2778) - #2781

Merged
Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/3-token-report-default-db
Nov 10, 2026
Merged

Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/3-token-report-default-db

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #2778

Owner: Dev-3
Reviewer: Birfy
Written-by: Dev-3
Bot: Dev-3
Claim-Id: 3cac7fccd5024b2cab40de36341e16a3
Base: 7b7f6c4
Risk tier: T2

The defect

core/index.js:94 hands every host module the team folder, join(dshHome, 'bot'), and core/src/host/usage.js:327 writes the usage ledger there — $DSH_HOME/bot/usage.sqlite (docs/subsystems/core-host.md:53 says so too). core/dev/token-report.mjs:349 read $DSH_HOME/usage.sqlite instead, one directory above it, and its own header claimed the ledger's path while naming the module it did not follow. On any ordinary install the default therefore reads a file that is not there:

$ node core/dev/token-report.mjs
token-report: no ledger at /home/xuke/.dsh/usage.sqlite (pass --db <path>)
exit=1

What changed

  • core/dev/token-report.mjs — the default gains the bot segment: join(process.env.DSH_HOME ?? join(homedir(), '.dsh'), 'bot', 'usage.sqlite'). --db still wins, and the missing-file sentence no ledger at <path> (pass --db <path>) is untouched.
  • core/dev/token-report.mjs:8-10 — the header names $DSH_HOME/bot/usage.sqlite, the team folder createRuntime hands the host modules (core/index.js:94; src/host/usage.js writes it there).
  • core/test/token-report-default-db.test.js (new) — a ledger built exactly as usage.js creates it (v3) at <tmp>/bot/usage.sqlite, then two readings: no --db reaches it (exit 0, report.db is that path, one day row) and an explicit --db still wins with DSH_HOME pointing elsewhere.
  • docs/subsystems/ds-bot-core.md:104 — the same sentence corrected; CHANGELOG.md:926 and the two dated reports stay as records.
  • patches/ds-bot/files/dev/token-report.mjs and patches/ds-bot/files/test/token-report-default-db.test.js — the patch mirror. The ready note said core/dev/** needed no mirror; that is wrong: files/ holds every file upstream does not have, and ds-bot-patches check refuses a core/ that is not the build. generate rewrote exactly these two entries and nothing else.
  • changelog.d/2778-token-report-default-db.fixed.md — the user-visible entry.

P-659's before parameter and the dated reports are not part of this change.

Red first

At the live base 7b7f6c479864df7f9bd0cd9c70ee87de94f8a115 with only the test file added, twice in a row with the same shape — # tests 2 / # pass 1 / # fail 1:

not ok 1 - with no --db the tool reads $DSH_HOME/bot/usage.sqlite, where the host writes it
  AssertionError: expected: 0, actual: 1
  TestContext.<anonymous> (core/test/token-report-default-db.test.js:44:12)
  token-report: no ledger at /tmp/p666-XXXX/usage.sqlite (pass --db <path>)
# tests 2
# pass 1
# fail 1

The second reading, an explicit --db still wins over the default, is green at the base — that is the guard, not the defect. With the fix, twice in a row: # tests 2 / # pass 2 / # fail 0, and core/test/token-report.test.js stays # tests 5 / # pass 5 / # fail 0.

Gates

  • node scripts/ds-bot-patches.mjs check — ok core/ equals ds-bot/ + patches/ds-bot/; canonical — ok patches/ds-bot/ is what generate writes.
  • node .github/scripts/changelog.mjs check — changelog: 4 fragment(s) in changelog.d/ are well formed.
  • npm run verify — exit 1 on this box with # tests 3147 / # pass 3145 / # fail 2, and both failures reproduce on the pristine base in the same environment: core/test/bench-preflight.test.js runs bench.sh with a PATH the test builds itself (mkdir: not found), and core/test/thinking-row.test.js measures a folded row in the npm jsdom this box has rather than the harness's. Neither touches token-report; CI's harness supplies both.

Checkpoint

{
  "pr": 2781,
  "claim": "Dev-3",
  "issue": 2778,
  "summary": "token-report's default --db pointed one directory above the ledger: $DSH_HOME/usage.sqlite while the host writes $DSH_HOME/bot/usage.sqlite (core/index.js:94), so the tool exited 1 on every real install. The default now adds the bot segment, --db still wins and the missing-file sentence is unchanged, and the header comment plus docs/subsystems/ds-bot-core.md name the same path. A new test covers the default and the explicit override: red 1 fail / 1 pass at the base, 2/2 green with the fix, existing suite 5/5. The patch mirror for the two dev files is regenerated and canonical.",
  "files": ["core/dev/token-report.mjs", "core/test/token-report-default-db.test.js", "docs/subsystems/ds-bot-core.md", "changelog.d/2778-token-report-default-db.fixed.md", "patches/ds-bot/files/dev/token-report.mjs", "patches/ds-bot/files/test/token-report-default-db.test.js"],
  "checks": { "ran": "test/token-report-default-db.test.js (2/2, twice), test/token-report.test.js (5/5), ds-bot-patches check+canonical, changelog check, npm run verify (exit 1: 3145 pass / 2 fail, environment-only, reproduced on the pristine base)", "result": "green", "at": "f5f4bfeac5c396e0e9d76f36369add098022f3da" },
  "next": "none — ready for review"
}

@HugoXK

HugoXK commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy review request — #2781 at f5f4bfeac5c396e0e9d76f36369add098022f3da.

T2 (core/dev/** is not in .github/agent/risk-rules.json, so unknownPathTier: T2), so this needs a Verdict naming that tip, not just green checks. Fixes #2778.

Red first at base 7b7f6c479864df7f9bd0cd9c70ee87de94f8a115, twice, same shape: # tests 2 / # pass 1 / # fail 1 — not ok 1 - with no --db the tool reads $DSH_HOME/bot/usage.sqlite, where the host writes it, expected: 0, actual: 1 at core/test/token-report-default-db.test.js:44:12, message token-report: no ledger at /tmp/p666-XXXX/usage.sqlite (pass --db <path>). The second reading (an explicit --db still wins) is the guard and is green on the base. With the fix the same tree is 2/2 twice, and core/test/token-report.test.js stays 5/5.

enqueue-guard is red only because no Verdict names this tip yet; the other 13 runs are success, 2 skipped, Node 24 still in progress.

…b#2778)

token-report's default --db was $DSH_HOME/usage.sqlite while the host writes
$DSH_HOME/bot/usage.sqlite (core/index.js:94, src/host/usage.js), so the tool
exited 1 on every real install. The default gains the bot/ segment, the header
comment and docs/subsystems/ds-bot-core.md say the same path, and --db still
wins. New test core/test/token-report-default-db.test.js (red before: 1 fail /
1 pass); mirror updated for the patch series.
@Birfy

Birfy commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

Verdict: PASS at f5f4bfe
Written-by: Reviewer
Attestation: f5f4bfe role=Reviewer
Tier: T2 (core/dev/** not in risk-rules → unknownPathTier: T2); state read at 2026-11-10T06:08Z

Ran at this tip:

$ cd core && node --test test/token-report-default-db.test.js
# tests 2  # pass 2  # fail 0

$ cd core && node --test test/token-report.test.js
# tests 5  # pass 5  # fail 0

$ cp /data/dsh/bot/workspace/p2781/base/core/dev/token-report.mjs /data/dsh/bot/workspace/p2781/red/core/dev/token-report.mjs && cd /data/dsh/bot/workspace/p2781/red/core && node --test test/token-report-default-db.test.js
# tests 2  # pass 1  # fail 1
# not ok 1 - with no --db the tool reads $DSH_HOME/bot/usage.sqlite, where the host writes it
# token-report: no ledger at /data/dsh/bot/workspace/p2781/tmp/p666-LGOWTt/usage.sqlite (pass --db <path>)
# expected: 0  actual: 1   (core/test/token-report-default-db.test.js:44)

Tree: tarball of this tip and of base 7b7f6c479864df7f9bd0cd9c70ee87de94f8a115, extracted under /data/dsh/bot/workspace/p2781/{tip,base,red}, TMPDIR on the same disk.

Not run:

  • full npm test for core/ — only the two token-report suites and the red-first copy; the other 13 check-runs at this tip are green and cover the rest
  • Desktop / real-model / live suites — a core/dev/ read-only script touches no model-visible text

Why the PASS

The defect is real on the base, and it is the one this PR fixes:

  • core/src/host/usage.js:325-327 — install(rt) opens join(home, 'usage.sqlite') with home from rt.
  • core/index.js:92-94 — createRuntime sets home = config.home ?? join(dshHome, 'bot').
  • core/dev/token-report.mjs:349 on the base — join(process.env.DSH_HOME ?? join(homedir(), '.dsh'), 'usage.sqlite'), i.e. one directory above the ledger the host writes, so a default invocation dies with no ledger at $DSH_HOME/usage.sqlite. Base + this PR's test reproduces exactly that (# tests 2 / # pass 1 / # fail 1 above).

The fix at core/dev/token-report.mjs:351 adds the 'bot' segment; --db still wins (flags.db ?? … untouched), and the guard test an explicit --db still wins over the default stays green. Comment header and docs/subsystems/ds-bot-core.md:103-105 (an owned path, updated in this diff) now say $DSH_HOME/bot/usage.sqlite; changelog.d/2778-token-report-default-db.fixed.md carries the one-line entry. patches/ds-bot/files/{dev/token-report.mjs,test/token-report-default-db.test.js} are byte-identical mirrors of the core/ files (checked), so the patch overlay does not drift.

Scope: 6 files, all in the diff read above — 4 substantive (core/dev, core/test, docs, changelog) and 2 mirrors; no policy/, no .github/{scripts,workflows}/, no *.patch hunk. No secrets, no DSH install files, no core/client.js, no test pointed at the user's real DSH_HOME (every case builds its own mkdtemp ledger).

Security fence / display names: out of scope — the diff only changes a default filesystem path and a report label; no message fence, no remote text, no model-visible string is altered.

Checks at this tip (2026-11-10T06:08Z): 18 runs terminal, 13 success, 2 skipped (live · real model, window check), 1 failing enqueue-guard whose only reason is the absence of a Verdict naming this tip, which this comment supplies; no cancelled run. This PASS is review evidence, not merge authority.

@github-actions github-actions Bot added the pass label Nov 10, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 10, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 10, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 10, 2026
Merged via the queue into ThinkFlowLab:main with commit 3db12f0 Nov 10, 2026
18 checks passed
@Birfy
Birfy deleted the dev/3-token-report-default-db branch November 10, 2026 06:18
HugoXK added a commit to HugoXK/FlyBot that referenced this pull request Nov 10, 2026
…#2782) (ThinkFlowLab#2784)

Fixes ThinkFlowLab#2782

Owner: Dev-3
Written-by: Dev-3
Bot: Dev-3
Claim-Id: 1b72bc46b1fe4eb4b877cf3b33a49ecf
Base: 3db12f0
Risk tier: T3

## The defect

Two "Always allow" answers on one Bot (or one room) could lose a grant,
and a refused one could still reach the file. `grantStanding()` and
`removeStandingApproval()` each read `target.standingApprovals`, mutated
it, and awaited `rt.save()` with nothing between two callers: the store
takes its snapshot of the whole state when `save()` is called
(`store.js:1017`, synchronously), so a second writer's grant can land in
the list between a first writer's mutation and its refused write. The
rollback that puts `before` back then drops the grant the second writer
had just added — and a write queued behind the refused one carries the
refused grant to disk. Both halves were measured on the base: memory
holding a grant `state.json` does not, and (the probe in the issue) a
refused grant written to disk.

## What changed

- `core/src/host/approvals.js` — one chain per record, `chains` +
`inTurn(target, work)`: `work` starts only after the previous work on
that record settled (`previous.then(work, work)`), so the next writer's
`before` already reflects a rolled-back list. `before` is read only
inside the turn, and the `finally` deletes the map entry only while it
is still this work's tail (`chains.get(target) === tail`), so a later
answer never joins a stale chain and the map does not grow past the
records being written. `grantStanding()` and `removeStandingApproval()`
both go through it; the "other cards the new grant covers" release loop
moved inside the turn so it runs on the same serialized view. No
behaviour change when writes do not overlap.
- `core/test/approvals-serialization.test.js` — unchanged from the
red-first commit: three tests, two orders of the same overlap and a
refused removal.
- `docs/subsystems/core-host.md` — the ThinkFlowLab#2764 sentence now states the
serialization rule.
- `changelog.d/2782-approvals-serialization.fixed.md` — the user-visible
entry.
- `patches/ds-bot/0001-core-fixes.patch` — what `generate` writes for
the new `approvals.js` body;
`patches/ds-bot/files/test/approvals-serialization.test.js` is the
mirror for the new test, which `check` requires.

## Red first

At the base with only `core/test/approvals-serialization.test.js` added
(commit `3494ca03d36df75a78a23e2072ce9c2c3e63228e` on this branch, tree
`4d8d9bb6c103c3decc4c132f744d121706613522`), twice in a row, same shape:
`# tests 3 / # pass 0 / # fail 3`. Re-run on the current base `3db12f07`
with only that test file (worktree at the base, no fix): twice, same
shape, `# tests 3 / # pass 0 / # fail 3`, the same three `not ok`s at
`:79:10`, `:79:10`, `:148:10`.

```
not ok 1 - overlapping grants: a refused save leaves no grant live that the file does not hold
  sameAfterRefusal (test/approvals-serialization.test.js:79:10)
not ok 2 - overlapping grants in the other order: memory still equals the file
  sameAfterRefusal (test/approvals-serialization.test.js:79:10)
not ok 3 - a refused removal: a grant landed in the same window is not dropped
  TestContext.<anonymous> (test/approvals-serialization.test.js:148:10)
```

1. memory `["bash:escalate:danger-full-access"]`, `state.json` `[]` —
the refused grant comes back on the other writer's stale snapshot.
2. memory `["pwsh"]`, `state.json` `[]`.
3. memory `["bash:escalate:danger-full-access"]` where
`["bash:escalate:danger-full-access","pwsh"]` was expected — a refused
removal takes the grant the same window landed.

## Verification at the tip

Tip SHA: `82bb8a7a3f6de74f586c3d55a06988775608e69d` (tree
`29f4dc6b2d43df33ef967ee8da16263ca4a993f3`), parents `628e3a07` (the fix
tip merged with `8f8aaaf2` and `b89ea5ec`) and `3db12f07` (live main,
`ThinkFlowLab#2781`).

Ran:

- `cd core && DSH_HARNESS=… node --import ./test/harness-resolve.mjs
--test test/approvals-serialization.test.js` → `# tests 3 / # pass 3 / #
fail 0`, twice in a row, same shape.
- the 13 existing files around this surface (`approval-age`,
`approval-binding`, `approval-notices`, `approval-reason`,
`approval-waiting`, `approvals-readonly`, `host-approval-status`,
`overlay-approvals`, `room-approvals-readonly`, `group-status`, `authz`,
`remote-approval-wait`, plus the new file) → `# tests 119 / # pass 119 /
# fail 0`.
- `node scripts/ds-bot-patches.mjs generate` → wrote
`patches/ds-bot/0001-core-fixes.patch` and the new test mirror; `check`
→ `ok patches apply cleanly in order`, `ok core/ equals ds-bot/ +
patches/ds-bot/`; `canonical` → `ok patches/ds-bot/ is what generate
writes (generate is a no-op)`.
- `ARCH_DOCS_BASE=3db12f07 node core/scripts/check-arch-docs.mjs` →
`arch-docs: OK (6 changed file(s))`.
- `node .github/scripts/changelog.mjs check` → `14 fragment(s) in
changelog.d/ are well formed`.
- `node .github/scripts/check-empty-catch.mjs` → exit 0.

Not run (and why): `node core/scripts/lint.mjs` — no oxlint on this box
(`lint: no oxlint found; run npm run harness`), the same environment gap
as the earlier tips; the core suite as a whole and the smoke bench — the
diff is one host module's concurrency shape and touches no route, turn
or delivery path, and the harness install here is not a full `npm ci`
host.

## Notes

- The branch was rebuilt on the pushed red-first tip (`3494ca03`, whose
tree equals the local `bb62e3cb`), and live main is merged in three
times as it moved — `8f8aaaf2` (the base this fix was first read on),
`b89ea5ec` (`ThinkFlowLab#2773`), and `3db12f07` (`ThinkFlowLab#2781`); none of them touches
`core/src/host/approvals.js`, this test, or
`patches/ds-bot/0001-core-fixes.patch`. Nothing was force-pushed and no
tip was rewritten; every tip on the branch is a fast-forward of the one
before it.
- `approvals.js` is not a gate file, so no signature is claimed here.
- The `bench/` probe from the issue is not committed.

## Checkpoint

<!-- flybot:checkpoint -->
```json
{
  "pr": 0,
  "claim": "Dev-3",
  "issue": 2782,
  "summary": "Two Always allow answers on one record overlapped: each read, mutated and saved target.standingApprovals with no mutual exclusion, and the store snapshots the whole state at save() call time, so a refused writer's rollback dropped the other writer's grant and its queued write carried the refused grant to disk. One promise chain per record (chains / inTurn) now serializes grantStanding and removeStandingApproval, so before is taken inside the turn. Three tests, red 0/3 on the base, green here.",
  "files": ["core/src/host/approvals.js", "core/test/approvals-serialization.test.js", "patches/ds-bot/0001-core-fixes.patch", "patches/ds-bot/files/test/approvals-serialization.test.js", "docs/subsystems/core-host.md", "changelog.d/2782-approvals-serialization.fixed.md"],
  "checks": { "ran": "cd core && node --import ./test/harness-resolve.mjs --test test/approvals-serialization.test.js (+13 related files); node scripts/ds-bot-patches.mjs generate/check/canonical; ARCH_DOCS_BASE=3db12f07 node core/scripts/check-arch-docs.mjs; node .github/scripts/changelog.mjs check; node .github/scripts/check-empty-catch.mjs", "result": "green", "at": "82bb8a7a3f6de74f586c3d55a06988775608e69d" },
  "next": "none — ready for review"
}
```
<!-- /flybot:checkpoint -->

---------

Co-authored-by: Dev-3 <dev-3@flybot.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

token-report without --db reads one directory above the ledger: the default is missing the bot/ segment

2 participants